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ABSTRACT 

In this work, we propose a probabilistic value-passing CCS 
(Calculus of Communicating System) approach to model 
and analyze a typical network security scenario with one at¬ 
tacker and one defender. By minimizing this model with re¬ 
spect to probabilistic bisimulation and abstracting it through 
graph-theoretic methods, two algorithms based on backward 
induction are designed to compute Nash Equilibrium strat¬ 
egy and Social Optimal strategy respectively. For each al¬ 
gorithm, the correctness is proved and an implementation is 
realized. Finally, this approach is illustrated by a detailed 
case study. 

Categories and Subject Descriptors 

C.2.0 [Computer-Communication Networks]: General— 
Security and protection 

General Terms 

Security 

Keywords 

Network security; Nash equilibrium strategy; Social optimal 
strategy; Reactive model; Probabilistic value passing CCS 

1. INTRODUCTION 

Modeling and analysis of network security has been a hot 
research spot in the network security domain. It has been 
studied from different perspectives. Among them are two 
main approaches, one based on game-theoretic metho ds , 
and one based on (probabilistic) process algebra |22| |4 . 
In the later 1990’s, game theoretic methods were introduced 
for modeling and analyzing network security [^. These 
methods consist in applying different kinds of games to dif¬ 
ferent network scenarios with one attacker and one defender 
[17| . Roughly speaking, static game is a one-shot game in 


which players choose action simultaneously. It is often used 
to model the scenarios in which the attacker and defender 
have no idea on the action chosen by the adversary (for 
instance the scenario of information warfare), and to com¬ 
pute the best strategy for players in a quantitative way [^. 
Stochastic game is often used to model the scenarios which 
involve probabilistic transitions through states of network 
systems according to the actions chosen by the attacker and 
the defender 10 1^. Markov game is an extension of game 
theory to MDP-like environments [^. It is often used to 
model the scenarios in which the future offensive-defensive 
behaviors will impact on the present action choice of attacker 
and defender [^. In Bayesian game, the characteristics 
about other players is incomplete and players use Bayesian 
analysis in predicting the outcome [^. A dynamic Bayesian 
game with two players, called Signaling game, is often used 
to model intrusion detection in mobile ad-hoc networks and 
to analyze Nash equilibrium in a qualitative way [^. On 
the other hand, as far as we know, (probabilistic) process 
algebra approach focus on verifying network security proto¬ 
cols. For example, in the earlier 1980’s, a simple version of 
the alternating bit protocol in ACPr (Algebra of Commu¬ 
nicating Processes with silent actions) was verified [^. For 
describing and analyzing cryptographic protocols, the spi 
calculus, an extension of the tt calculus, was designed [^. 
Recently, a generalization of the bisimilarity pseudo-metric 
based on the Kantorovich lifting is proposed, this metric al¬ 
lows to deal with a wider class of properties such as those 
used in security and privacy [^. 


In this paper, we propose a probabilistic value-passing CCS 
(PVCCS) approach for modeling and analyzing a typical 
network security scenario with one attacker and one de¬ 
fender. A network system is supposed to be composed of 
three participants: one attacker, one defender and the net¬ 
work environment which is the hardware and software ser¬ 
vices of the network under consideration. We consider all 
possible behaviors of the participants at each state of the 
system as processes and assign each state with a process de¬ 
scribing all possible interactions currently performed among 
the participants. In this way we establish a network state 
transition model, often called reactive model in the litera¬ 
ture [^, based on PVCCS. By minimizing this model with 
respect to probabilistic bisimulation and abstracting it via 
graph-theoretic methods, two algorithms based on backward 







induction are designed to compute Nash Equilibrium Strat¬ 
egy (NES) and Social Optimal Strategy (SOS) 

[14| 1^ respectively. The former represents a stable strat¬ 
egy of which neither the attacker nor the defender is willing 
to change the current situation, and the latter is the pol¬ 
icy to minimize the damages caused by the attacker. Eor 
each algorithm, the correctness is proved and an implemen¬ 
tation is realized. This approach is illustrated by a detailed 
case study on an example introduced in [^. The example 
describes a local network connected to Internet under the 
assumption that the firewall is unreliable, and the operating 
system on the machine is insufficiently hardening, and the 
attacker has chance to pretend as a root user in web server, 
stealing or damaging data stored in private file server and 
private workstation. The major contributions of our work 
are: 


• establish a reactive model based on PVCCS for a typi¬ 
cal network security scenario which is usually modeled 
via perfect and complete information games. 

• minimize the state space of network system via proba¬ 
bilistic bisimulation and abstract it via graph-theoretic 
methods. This allows us to reduce the search space 
and hence considerably optimize the complexity of the 
concerned algorithms. 

• propose two algorithms to compute Nash Equilibrium 
and Social Optimal strategy respectively. The nov¬ 
elty consists in combing graph-theoretic methods with 
backward induction, which enables us on the one hand 
to increase reuseness and on the other hand to make 
the backward induction possible in the setting of some 
infinite paths. 


2. PRELIMINARIES AND REACTIVE MODEL 
BASED ON PVCCS 


2.1 Graph theory 

We firstly recall some notions of graph theory: Strongly Con¬ 
nected Component (SCC), Directed Acyclic Graph (DAG) 
and Path Contraction iHli- 


SCC of an arbitrary directed graph form a partition into 
subgraphs that are themselves strongly connected (it is pos¬ 
sible to reach any vertex starting from any other vertex by 
traversing edges in the direction). 


DAG is a directed graph with no directed cycles. There are 
two useful DAG related properties we used in our paper: (1) 
if if is a weakly connected graph, H' is obtained by viewing 
each SCC in H as one vertex, H' must be a DAG; (2) if H 
is a DAG, H has at least one vertex whose out-degree is 0. 


Path Contraction Let e = xy he an edge of a graph 
H = (y,E). H/e is a graph {V\E') with vertex set V' := 
(y\{a:,y}) U {ve}, and edge set E' {vw E E \ {v,w} D 
{x,y} = 0} U {veW I xw e E\{e} or yw G E\{e}} (Eigure 
[D. Path contraction occurs upon the set of edges in a path 
that contract to form a single edge between the endpoints 
of the path after a series of edge contractions. 




Figure 1: Edge contraction 


Note that our method can filter out invalid Nash Equi¬ 
librium strategies from the results obtained by traditional 
game-theoretic methods. Eor instance, in the example in¬ 
troduced in [^, three Nash Equilibrium strategies obtained 
ultimately by game-theoretic approach methods, while only 
two of them obtained by our method: we filter out the invalid 
Nash Equilibrium strategy from the results in [^. Note 
also our method can be applied to other network security 
scenarios. Eor example, the proposed reactive model can 
be extended conservatively to a generative model based on 
PVCCS. In this way we provide a uniform framework for 
modeling and analyzing network security scenarios which 
are usually modeled either via perfect and complete infor¬ 
mation games or via perfect and incomplete games. How¬ 
ever, for the limited space of this paper, we will focus on the 
reactive setting for the conciseness and easier understanding 
of this work. 

In the remaining sections, we shall review some notions 
of graph theory and establish the reactive model based on 
PVCCS (Section 2); present the formal definitions of NES 
and SOS in this model, as well as the corresponding algo¬ 
rithms and their correctness proofs (Section 3); then illus¬ 
trate our method by a case study (Section 4); fianlly, discuss 
the conclusion (Section 5). Appendix shows proofs of theo¬ 
rems, tables referred to the case study and a notation index. 


2.2 PVCCSr 

PVCCSi^ is a reactive model for Probabilistic Value-passing 
CCS, proposed based on the reactive model for probabilistic 
CCS [^. 

Syntax: Let A be a set of channel names ranged over by 
a, and A be the set of co-names, i.e., A = {a | a G A}, and 
a = a by convention. Label = AU A. Var is a set of value 
variables ranged over by x and Val is a value set ranged 
over by r'. e and b denote value expression and boolean 
expression respectively. The set of actions, ranged over by 
a. Act = {a{x) \ a G A} U {a(e) | a G A} U {r}, where r 
is the silent action. JC and X are a set of process identifiers 
and a set of process variables respectively. Each process 
identifier A G /C is assigned an arity, a non-negative integer 
representing the number of parameters which it takes. 

Pr is the set of processes in PVCCSi^ defined inductively as 
follows, where P, Pi are already in Pr: 

Pr I I A|P 2 I P\R 

iei jeJ 

I ifh then Pi else P2 \ A(x) 
a ::=a(x) | a(e) 






where a G Label, RCA. J are index sets, and Vz G /, 
Pij G (0,1], = 1, and a^ ^ aj if i ^ j. Y. and Y are 

3^1 

summation notations for processes and real numbers respec¬ 
tively. Furthermore, each process constant A{x) is defined 
recursively by associating to each identifier an equation of 

def 

the form A(x) = P, where P contains no process variables 
and no free value variables except x. 

Nil is an empty process which does nothing; Y Y 

i Ei I jEiJ 

is a summation process with probabilistic choice which means 
if performs action Pij will be chosen to be proceed with 
probability pij^ for example, [ 0 . 2 ]a.Pi + [ 0 . 8 ]a.P 2 + [l]j3.P3 
is a process which will choose process Pi with probability 
0.2 and P 2 with probability 0.8 if performs action a, or will 
choose P 3 with probability 1 if performs action here ai 
stands for an action prefix and there are two kinds of pre¬ 
fixes: input prefix a(x) and output prefix a(e). If J is a 
singleton set, then we will omit the probability from the 
summation process, such SiS YY [^o^i.Pij will be written 

i^I j^J 

as if both I and J are singleton sets, then the 

iei 

summation process is written as a.P; P 1 IP 2 represents the 
combined behavior of Pi and P 2 in parallel; P\R is a chan¬ 
nel restriction, whose behavior is like that of P as long as 
P does not perform any action with channel a G PUP; 
ifh then Pi else P 2 is a conditional process which enacts Pi 
if b is true, else P 2 . 

Semantics: The operational semantics of PVCCSi^ is de¬ 
fined by the rules in Tablewhere P Q describes a tran¬ 
sition that, by performing an action a, starts from P and 
leads to Q with probability p. Mapping chan : Act ^ A, 
i.e., chan{a{x)) = chan{a{e)) = a. And P{e/a:} means 
substituting e for every free occurrences of x in process P. 

By convention, if Pi P 2 and P 2 P 3 , then we use 
Pi P 3 to represent multi-step transition. 

Probabilistic Bisimulation: We recall the definition of 
cumulative probability distribution function (cPDF) which 
computes the total probability in which a process derives a 
set of processes, p is the powerset operator and we write 
Pr/P to denote the set of equivalence classes induced by 
equivalence relation IZ over Pr. 

Definition 2.1. p : (Pr x Act x p(Pr)) ^ [0,1] is the 
total funetion given by: Vo G Act, VP G Pr, VC C Pr, 

ti{P,a,C) = ^{p\p Ai Q, Q e C}. 

Definition 2.2. An equivalence relation IZ C Pr x Pr 
is a probabilistic bisimulation if (P, Q) ^ IZ implies: VC G 
PvjlZ, Vo G Act, p{P,a,C) = p{Q,a,C). 

P and Q are probabilistic bisimilar, written as P ~ Q, if 
there exists a probabilistic bisimulation IZ s.t. PIZQ. 

2.3 Modelling for Network Security based on 
PVCCSr 


ComModel focuses on modeling the network security sce¬ 
nario modeled usually via perfect and complete information 
game: a network system state considers the situations of 
attacker, defender and network environment together; the 
participants act in turn at each state and the interactions 
among the participants will cause the network state transi¬ 
tion with certain probability; each state transition produces 
immediate payoff to attacker and defender, and the former 
(positive values) is in terms of the extent of damage he does 
to the network while the latter (negative values) is mea¬ 
sured by the time of recovery; the future offensive-defensive 
behaviors will impact on the action choice of attacker and 
defender at each state. Nash Equilibrium strategy repre¬ 
sents a stable plan of action for attacker and defender in 
long run, while the Social Optimal strategy is a policy to 
minimize the damage caused by attacker. 

Assuming S is the set of network system state, ranged over 
by Si, z G /, / is an index set; action sets of attacker and 
defender are A“ and A^ respectively, u, v represent the gen¬ 
eral values, A“(si) C A“ is the action set of attacker at Si, 
as well as A^(si) C A^ is that of defender; state transition 
probability is a function p : S x A^ x A^ x S ^ [O 5 1]? and 
immediate payoff associated with each transition is a func¬ 
tion r : S' X A“ X Ri X R 2 , where R is the real number 

set, and we use index to distinguish the first and the second 
element, and : S' x A“ x ^ Ri represents the immedi¬ 
ate payoff of attacker, while : S' x A“ x A^ ^ R 2 is that 
of defender. 

ComModel, a model based on PVCCSr, is used to model¬ 
ing for the network security scenario depicted as above. The 
processes represent all possible behaviors of the participants 
in network system at each state. Each state is assigned with 
a process depicting all possible interactions currently per¬ 
formed among the participants. Then we establish a network 
state transition system based on the process transitions. 

In ComModel, the channel set A = {Attc, Defd,Tella^ 
Telld}, Label = A U A U {Log} U {Rec}. The value set 
Val = A^ U A^ U T, where T C R x R. Var is the set 
of value variables. Act is the union of behavior sets of the 
three participants {AcP,Act^ and Act^) defined as follows: 

Act =AcP U Act^ U Act^ 

AcP —{Attc{y) I V G A“} U {Tella{x) \ x G Var} 

Act^ —{Defd(v) I V G A^} U {Telld{x) \ x G Var} 

Act^ —{Attc(x) I X G Var} U {Defd{x) \ x G Var} 

U {Tdra{x) I X G Var U A^} 

C{LWd{x) I X G VarU A“} 

U {Log{x, y) I X G A“ U Var, y G A"^ U Var}. 

U {Rec{r{s, u,v)) | s G S', zz G A“, v G A"^} 

Eigure shows one interaction among the participants at 
state s. Attc(u) means attacker takes attack zz, similar 
to Defd{v) for defender; Attc{x) (or Defd{x)) means net¬ 
work environment is attacked (or is defended); Telld{x) (or 
Tella{x)) means network environment informs defender (or 
attacker) the action chosen by attacker (or defender); Telld{x) 
(or Tella{x)) means defender (or attacker) is informed that 
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Table 1: Operational semantics of PVCCSi? 
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Figure 2: Interactions among participants at state s 


path contractions on TS and obtained a new graph named 
as ConTS without information loss as follows: 


Definition 2.3. ConTS is a tuple {V,E,L) 

• V = {Gi I Gi is the proeess we assign to state s^} 

• E — {{Gi^Gj) I ranged over Cij, if there exists a multi- 

/ ^ GmGLog(u,v)[l]Rec(r(si,u,v))[p(s^,u,v,Sj)] 

transition Gi Gj| 

• Li^eij) — \^{^L Acti^^ij) 1 L TranPi^^ij) 1 Lweipi^eij) | Oij G E^ 


attack (or defense) has happened; Log{x,y) means the net¬ 
work environment writes the values of x and y into a log 
hie, where x and y is used to receive the values of attack 
and defense respectively; Rec{f{s^u^v)) stands for the net¬ 
work environment records the immediate payoff to attacker 
and defender if they choose u and v at state s respectively. 


— aetion pair: LAct^Gj) — {u^ v) 

— transition probability: LTranp{Gj) — p{si, u, v, Sj) 

— weight pair: LweiP^Gj) — r{si^u^v) 

* Lweip(Gj) = r^(s^,u,v) 

* Lwexpieij) = r‘^(si,u,v) 


The processes describing all possible behaviors of the partic¬ 
ipants at state denoted by pA-, pD^ and pN-, are dehned 
as follows: 

pAi Yi AMi{u).Tellaiy).Nil 

uEA^ (si) 

pDA= Telldix). Y 'Defd{v).Nil 

veA^(si) 

pNi Attcix).T^(x).Defd{y).Tdmy).Tr^(x,y) 
Tri(x,y)^ Log{u^v).{if {x — u^y — v) then 

ueA^{si) 

veAd(sP 

y^[p(s„ u, V, Sj)]Rec{r{si, u, v)).{pAf [pD^ [pN^) 
iei 

else Nil) 


Lweip{Gj) = L'^f,i^p{eij) + \L'^^ip{eij)\ denotes the sum of 
absolute weight pair of By convention, in any network 
security scenario, for any e, e' G E, if L^^^p{e) > L^g^p(e') 
then L^g^p(e) < L^g^p(e'). 

3. ANALYZING PROPERTIES AS GRAPH 
THEORY APPROACH 

We hrstly introduce the dehnitions of Nash Equilibrium strat¬ 
egy (NES) and Social Optimal strategy (SOS) in our model, 
and then we illustrate the algorithms proposed to comput 
NES and SOS respectively. 

3.1 NES and SOS 

Definition 3.1. G V, an execution ofGi in ConTS, 
denoted by TVi, is a walk (vertiees and edges appearing alter¬ 
nately) starting from Gi and ending with a eyele, on whieh 
every vertex’s out-degree is 1. 


The process assigned to each state Si is dehned as 

Gi =\pA^\pD^[pN^)\R,R= {Attc,Defd,Telia,Tdld} 

We get the network state transition system, TS for short, 
based on process transitions. Minimizing TS by shrinking 
probabilistic bisimilar pairs of states. We conduct a series of 


According to the dehnition of execution, tt* is in the form 
of GieijGj...(Gk---GieikGk) which is ended by a cycle starting 
with Gfc, where Gi and Gk may be the same node, tt^ can 
be written as 7r| if e is the hrst edge of tt^; 7Ti[j] denotes the 
subsequence of TVi starting from Gj, where Gj is a vertex on 

7Tt. 
























Definition 3.2. The payoff to attacker and defender on 
execution Hi, denoted by PF^{7Vi) and PF^{7Vi) respectively, 
are defined as follows: 

PF'"{7T^) = Lweipi^ij) + ^ • LTranP{etj) • PF'"{7T^[j]) 

= L%^ip{eij) + /3 • LTranP{,e^j) • PF'^{H^[j]) 

where (3 G (0,1) is a discount factor. The sum of absolute 
payoff on Hi of attacker and defender is denoted as PF^{Hi), 
and PF^{H^) = PF^{Hf) + \PF^{Hf)\. 

Theorem 3.1. \/Gi G V, Hi is an execution ofGi, PF^{Hi) 
and PF^(Hi) are converged. 

Proof. Based on the definition of payoff on an execu¬ 
tion of Gi and limiting laws, we show the proof details for 
PF^{Hi) in Appendix. The proof for PF^{Hi) is similar. □ 

Nash Equilibrium Execution and Social Optimal Execution 
are defined coinductively as follows: 

Definition 3.3. Hi is Nash Equilibrium Execution (NEE) 
of Gi if it satisfies: 

PE“(7rJ = max {Lweip{eij) + (3 • LTranp{eij) ■ PF'"{hj)} 

CijEE' (Gi) 

PF'^{Hi) = max {Lweipi^tj) + ^ • LTranP{eij) • PF'^(Hj)} 

eijeE^(Gi) 

where Hj is NEE of Gj, e is the first edge of Hi, E^{Gi) = 
{e! G E(Gi) I L\f,fe) — including e, and E'{Gi) — 

{arg max {L^e^p(e') + {3 • LTranp{e) • PF'^{Hj)},ye" G 

e'eE^,,(Gi) 

E{Gf)}. 

Definition 3.4. Hi is Social Optimal Execution (SOE) of 

Gi, if it satisfies: 

PF^{Hi) = min {Lweip{etj) + 13 • LTranP{eij) • PF^{Hj)} 

eijCE(Gi) 

where Hj is SOE of Gj 

Definition 3.5. Strategy is a sequence consisting of ac¬ 
tion pair (one from attacker and one from defender) at each 
state. 

Definition 3.6. Nash Equilibrium Strategy (NES) is a strat¬ 
egy of which every Gi’s execution based on is NEE of Gi. 

Definition 3.7. Social Optimal Strategy (SOS) is a strat¬ 
egy of which every Gi’s execution based on is SOE of Gi. 

3.2 Algorithms 

The way to compute NES (or SOS) in ConTS is to find 
a spanning subgraph of ConTS satisfying following condi¬ 
tions: 

A. Each vertex’s outdegree is 1; 

B. Each vertex’s execution in this subgraph is its NEE (or 
SOE). 


Eor backward inductive analysis, we firstly find SCO of ConTS 
based on Tarjan’s algorithm and construct Abstrac¬ 
tion (Abs for short) by viewing each SCC as one vertex. 
V(Abs) denotes the vertex set of Abs ranged over by D. 
Abs is a DAG, and we rename D with Leave if its out-degree 
is 0, else with Non-Leave. By convention, \/D G V(Abs), 
V{D) = {Gi E V \ Gi belongs to the SCC represented by 
D}. 

Definition 3.8. VD G V(Abs), the priority of D, de¬ 
noted by prior(D), is defined inductively: 

(1) prior(D) — n, if D is a Leave, and n is the size of 

Y(Ahs). 

(2) prior(D) = inm{prior(D') — 1 \ D' is any direct succes¬ 
sor of D in Abs} 

Definition 3.9. D depends on D' if D' appears in one 
of the paths starting from D in Abs. 

Theorem 3.2. Ifprior{D) < prior{D') then D' does not 
depend on D. 

Proof. We prove it by contradiction: if D' depends on 
D, then D appears in one of the paths starting from D' in 
Abs, so we have prior{D') = mm{prior{D") — 1 | D" is 
any direct successor of D' in Abs} < prior(D), contradic¬ 
tion. □ 


If D does not depend on D', then computing NES/SOS of 
D' has no impaction on computing NES/SOS of D. To find 
NES/SOS of D is to find NEE/SOE of all G^ G V{D). 

The algorithms for computing NES and SOS, denoted as 
AlgNES() and AlgSOS() respectively, are both based on 
backward induction. The framework of AlgNES() is as 
follows: 

(1) Compute priority of each vertex D in Abs; 

(2) Compute NES for Leave firstly, then compute backward 
inductively for Non-Leave. 

The framework of AlgSOS() is similar. 

Pseudo code of AlgNES() is shown in Algorithm]^ 

Data: Abs 
Result: NES of Abs 
NES(Ahs) ^ 0; 
for D G V(^Absj do 
I prior(D) p- ComputePrior(D); 
end 

List C P- list of 'V(Abs) in descending order on priority; 
pointer p ^ C-, 

while p is not the tail of C do 
D G- p.data] 

while prior(D) is the highest in C, do 
iVE;5(Abs)U ^ NESinLeave(D); 
p P- p.next] 

D p- p.data] 

end 

NES(Ahs)U ^ NESinNonLeave(D); 

p G- p.next] 

D G- p.data] 

end 


Algorithm 1: Pseudo code of AlgNES() 




NES/SOS for Leave 

The key point of computing NES (or SOS) for Leave D is to 
find a cycle in D satisfying conditions A and B as above. 

NES in Leave: The method of finding NES for Leave D is a 
value iteration method, denoted as NESinLeave(D). The 
value function is Backlnd(Gi) which returns some edge e of 
Gi and RefN(Gi) is used to refresh the value of the weight 
pair for each edge of Gi, WGi G V{D). 

As the narrative convenience, we introduce some auxiliary 
symbols: Ve G E{Gi), the weight pair initializes with Lo(e) = 
Lweip{G), and Ln{e) — (L“(e), L^(e)) is used to keep the 
new weight pair of e obtained by RefN(Gi) on the nth iter- 
ation; VG, e V{D), Ppn{Gi) = {Pp'^{Gi),Ppi{Gi)), initial- 
ized with Ppo(G^z) = (0, 0), is used to keep Ln(e), where e is 
the result of Backlnd(Gi) on the nth iteration. The itera¬ 
tive process will be continued until \/Gi G V(P), Ppn{Gi) = 
Ppn+liGi). 

The framework of NESinLeave(T)) is as follows: 

(1) Value iteration initializes with Backlnd(Gi), where for 
each Gi G V(P), the weight pair of e G E{Gi) is Lo(e). 
Assuming e is the result obtained by Backlnd(Gi), then 
Ppo(Gz) = Lo(e); 

(2) Loops through the method RefN(Gi) and Backlnd(Gi) 
by order until \/G^, Ppn+l{G^) = Ppn{G^)^, 

(3) \/Gi, execute Backlnd(Gi). The cycle obtained is what 
we want. 

Rules of method Backlnd(Gi) on the nth iteration, n > 0: 

(1) Let E' = E{G^)] 

(2) If 3ei,e2 G E' satisfying refresh E' 

by filtering the edge e ^ arg max Ln{e); 

eG{ei,e2} 

(3) Refresh E' by keeping edge e = argmaxL“(e) 

eeE' 

(4) Return e. 

Rules in method RefN(G^) on the (n+l)th iteration, n > 0: 

(1) Ve G E{Gi), compute its Ln+i{e) componentwise by fol¬ 
lowing formula: 

Ln+l(yG-ij) — LweiPi^^ij) T ^ ‘ L TranP(^G-ij) • Ppn{G j) 

(2) Keep Ln+l{e^j), \/e^J G E{Gi)] 

Pseudo code of NESinLeave(), Backlnd() and RefN() 
are shown in Algorithm and respectively. 

Data: Leave of Abs: D 
Result: NES of D 

Label Gi G V(-D) with NonConducted; 

NES{D) ^ 0 ; 

while 3 Gi is NonConducted do 
e ^ BackInd(Gd; 

Ppo{G,) ^ Lo{ey, 

Label Gi with Conducted; 

end 

while 3 Gi Ppn(Gi) ^ Ppn+i{Gi) eomponentwise do 

RefN(Gd ; 
e P- BackInd(Gd; 

PPn+l{Gi) p- Ln+l{e)] 

end 

NES{D) ^ {e\e ^ BackInd(Gd, G, G V(L>)}; 

Algorithm 2: Pseudo code of NESinLeave() 


Data: G, G VfD) 

Result: edge e G E(Gi) 
create E' p- E{Gi)\ 

while Vei,e2 G E' with L^Sei) = LTde2) do 
I E' ^ E'\{e I e / argmax {L^(e)}}; 

I ee{ei,e2} 

end 

e ^ argmax{L“(e)}; 

eeE' 
return e; 

Algorithm 3: Pseudo code of Backlnd() 

Data: G, G VfD) 

Result: Ln+i{eij), ^eij G E(Gi) 

Label all G E{Gi) with NonRef; 
while 3 eij is NonRef do 

i ^ WeipGij) T ‘ ^ TranPGij) ' ^Pn^Gj)'i 
^n+lGO ^ WeipGij) T ' ^TranPGij) ' ^Pn(Gj)', 

Label with Ref; 

end 

Algorithm 4: Pseudo code of RefN() 


SOS in Leave: The method SOSinLeave(P) used to find 
SOS for Leave D is also a value iteration method. The value 
funetion is LocSoOp(Gi) which returns some edge e of Gi 
and RefS(Gi) is used to refresh the absolute sum value of 
the weight pair for each edge of Gi, \/Gi G V(P). 

Here are some other auxiliary symbols for convenience: Ve G 
E(Gi), its sum of absolute weight pair initializes with Lo(e) = 
L^g^p(e), and L^(e) is used to keep the new sum of absolute 
weight pair of e obtained by RefS(GJ on the nth iteration; 
Psn{Gi) initialized with Pso{Gi) = 0, is used to keep L^(e), 
where e is the result of LocSoOp(Gi) on the nth iteration. 
The iterative process will be continued until WGi G V{D), 
PSniGi) — PSn+l{Gi). 

The framework of SOSinLeave(P) is as follows: 

(1) Value iteration initializes with LocSoOp(Gi), where for 
Gi G V{D), the sum of absolute weight pair of e G E{Gi) is 
Ln(e). Assuming the result obtained by LocSoOpfGd is e, 
then Pso(G^) = L^(e); 

(2) Loops through the method RefS(Gi) and LocSoOp(GJ 
by order until VG^, Psn+iiGi) = Psn{Gi)', 

(3) VGi, execute LocSoOp(Gi). The cycle obtained is what 
we want. 

Rules of method LocSoOp(Gi) on nth iteration, n > 0: 

(1) Compare Ln(e), Ve G P(Gi); 

(2) Return edge e = arg min ^{L;^(e)}. 

Rules of method RefS(Gi) on (n+l)th iteration, n > 0: 

(1) Veij G E(Gi), compute its Ln+i{eij) by following formula: 

Ln+l^^ij) — ^Weip{^ij) + ‘ LpranPi^ij) ' PSn{Gj) 

(2) Keep Ve € E{Gi)\ 

Pseudo code of SOSinLeave(), LocSoOp() and RefS() 
are given in Algorithm and respectively. 

NES/SOS for Non-Leave 

NES of Non-Leave: For Non-Leave vertex D in Abs, the 
method of computing its NES is NESinNonLeave(D) and 
its framework is as follows: 





Data: Leave of Abs: D 
Result: SOS of D 

Label Gi G V(-D) with NonConducted; 

SOS{D) ^ 0; 

while 3Gi is NonConducted do 

e -h- LocSoOp(G^); 

Pso(Gi) ^ L®(e); 

Label Gi with Conducted; 

end 

while 3G, Psn{G^) / Psn+i{G^) do 
RefS(G*) ; 
e ^ LocSoOp(G^); 

P5„+i(G,)^L®+i(e); 

end 

SOS(D) ^ {e\e ^ LocSoOp(G,), G^ G V(L>)}; 

Algorithm 5: Pseudo code of SOSinLeave() 

Data: G^ G V(D) 

Result: edge e G E{Gi) 

while 3e G E{Gi) is not eompared do 

I e' arg min{L^(e)}; 

I e^E 

end 

return e'; 

Algorithm 6: Pseudo code of LocSoOp() 

Data: G, G V(D) 

Result: L^_j_^{eij),\/eij G E{Gi) 

Label all eij G Gi with NonRef; 
while 3eij is NonRef do 

L^_l_]^{eij) ■<— L^^^p{eij) + (3 • LTranP{&ij) • PSn{Gj)] 

Label e^ with Ref; 

end 

Algorithm 7: Pseudo code of RefS() 


(1) if the size of V(D) is more than 1, we will pre-process 
D with method PrePro(D) firstly, then get its NES by 

NESinLeave(D); 

(2) if V{D) = {Gi} for some Gi G V, then the NES of D is 
the result obtained from Backlnd(Gi) directly. 

Rules in method PrePro(D) are as follows: 

(1) D' is one direct successor of D in Abs, and if the edge 
e connecting D and D' is contributed by the connection be¬ 
tween Gi G V{D) and Gj G V(D'), then Lo(eij) = Lweip(e d+ 
/3 • LTranp{eij) ' PF^TTj) componcntwisc, where TVj is the nash 
equilibrium execution of Gj] 

(2) Change e to be the self-loop edge of Gi. 

Pseudo code of NESinNonLeave() and PrePro() are shown 
in Algorithm and Algorithm respectively. 

Data: Non-Leave: D 
Result: NES of D 
NES{D) ^ 0; 

if the size ofV(D) is hiqqer than 1 then 
I D' ^ PrePro(D); 

I NES{D) ^NESinLeave(D'); 

else 

I NES(D) ^BackInd(Gd, if V(L>) = {G^}] 

end 

Algorithm 8: Pseudo code of NESinNonLeave() 


SOS of Non-Leave: The method SOSinNonLeave(D) 

computing SOS for Non-Leave D is identical to NESin- 
NonLeave(T)) except for the preprocessing method Pre- 
ProS(D). The computing steps of PreProS(D) are as fol- 


Data: Non-Leave: D 
Result: new D' 

E' ^ E{G,),G, G V(D); 

while 3eij G E' with endpoint Gj ^ V(-D) do 

^^eipGij) T ■ ^TranPGij) ' Pp°'{Gj)\ 

^oGij) ^WeipGij) + ^ ' ^TranPOij) • Pp^(Gj); 
Change e^ to be self-loop edge of G^; 

end 

D' ^ (V(D),E')- 
Return D'; 

Algorithm 9: Pseudo code of PrePro() 


lows: 

(1) D' is one direct successor of D in Abs, if the edge e 
connecting D and D' is contributed by connection between 
Gi G V(-D) and Gj G V(-D ), then Tq (cij) = L^^jp{eij^ + [3 • 
LTranp{eij) ‘ PF^ (iTj), whcrc TTj IS social optimal execution of 

(2) Change e to be self-loop edge of Gi. 

Pseudo code of SOSinNonLeave() and PreProS() is shown 
in Algorithm 1 10| and Algorithm ] 11 [ respectively in Appendix. 

Data: Non-Leave: D 
Result: SOS of D 
SOS(D) ^ 0; 

if the size of V(D) is hiqqer than 1 then 
I D' ^ PreProS(D); 

I SOSiD) ^SOSinLeave(D'); 

else 

I SOS(D) ^LocSoOp(Gd, if V(D) = {GR; 

end 

Algorithm 10: Pseudo code of SOSinNonLeave() 


Data: Non-Leave: D 
Result: new D' 

E' ^ E{Gi),G^ G V(D); 

while 3eij G E' with endpoint Gj ^ V{D) do 

I ^0 Gv) ^WeipGh) T ^ ' ^TranPGij) ' Ps(Gj); 

I Change e^ to be self-loop edge of G^; 

end 

D' ^ iV{D),E')- 
Return D' 

Algorithm 11: Pseudo code of PreProS() 


3.3 Correctness of Algorithms 

Correctness of NESinLeave() 

Inspired by a technique in dynamic programming which is 
called value-iteration [^[^, Backlnd(H) is formalized as 
a mapping a : V{D) ^ R x R, on kth iteration, ctkiGi) = 
{cFk{GiY,cFk{G^Y) = Ppk{G^). RefN() defines a set of 
vertex {Gi{ak) \ Gi{ak) denotes Gi with e^ whose weight 
pair is refreshed by the rule componentwise: Lk+i{eij) = 
Lweip{eij) + P • Lpranpieij) ‘ crfc(Cj)}. According to the rules 
of NESinLeave(D), ak-\-i{Gi) = Ppi{Gi{ak)) for any Gi G 
V(D). It is convenient to define the shorthand operator no¬ 
tation {Ta){Gi) = Ppi{Gi{a)), that is Tcr^ = Ckpi- 

Lemma 3.1. For any Gi G V(D), we have 
I (7k{G^Y - Tak{G^Y \< max | Ll{e) - Lfc+i(e) | 

eeE(Gi) T V / I 

-TakiGif \< max | p(e) - LAi(e) | 

eeE{Gi) 






Proof. We prove by contradiction for the first inequality 
in details. According to the rules in Backlnd(Gi), we need 
to consider all possible results obtained by Backlnd(Gi) 
on kth and (k+l)th iteration respectively. The details are 
shown in Appendix. The proof for the second inequality is 
similar. □ 


Theorem 3.4. If D is a Leave of Ahs, then the result 
obtained by SOSinLeave(D) is SOS of D. 

Proof. The proof is similar to that of Theorem |3.3| □ 


Lemma 3.2. T is a eontraetion. 

Proof. For any real vector G J is an in dex set, 
let II ||oo= maxj \ xj \. According to Lemma [3J^ then we 
have 

II Ta^+i - Tal ||oo = max | T(Jk+i{GiY - T(Jk{GiY \ 

GiEV 

< max max | Ll+ 2 {eij) - Ll+i{eij) \ 

E V E-F(G j) 

- I ^k+i{G,r - <Jk{G,r I 

_ Q \\ CL a I I 

— P’ N ^/c+1 ~ Noo 

similar proof for || — Tcrf ||oo< fd- || crt+i — l|oo- 

Therefore, we claim that 3cr*, satisfying □ 


Theorem 3.3. If D is a Leave of Abs, then the result 
obtained by NESinLeave(D) is NES of D. 


Correctness of AlgNES() and AlgSOS() 


Theorem 3.5. The results obtained from AlgNES(Abs) 
and AlgSOS(Abs) are NES and SOS of Abs respeetively. 

Proof. We prove the correctness of AlgNES(Abs) in 
details. Prove inductively on priority of vertex D in Abs. 

(1) If D is a Leave, we need to prove the resul t of NESin- 
Leave(D) is NES of D, according to Theorem |3.3[ trivial; 

(2) For Non-Leave D, and we assume prior(D) = prior{D') — 
1, by induction hypothesis, D' has got its NES by Al- 
gNES(). If V{D) = {Gi} for some Gi G V, according to 
the dehnition of NEE and rules of Backlnd(), the proof is 
trivial; if the size of V(D) is bigger than 1, according to the 
theorem |3.3| trivial. □ 


Proof. We need to prove two issues: 

1. NESinLeave(T)) is terminated. 

2. The execution of Gi, WGi G V{D), based on the result of 
NESinLeave(D) is its nash equilibrium execution. 

The details are shown in Appendix. □ 


Correctness of SOSinLeave() 

The way to prove the correctness of SOSinLeave() is sim¬ 
ilar to that of NESinLeave(). We will give the outline of 
the proofs. 


We can formalize LocSoOp(D) as a mapping a' : V{D) 

R, so on kth iteration, we have a'j^{Gi) = Psk{Gi). RefS() 
defines a set of vertex {Gi^a'^) \ Ci(cr^) denotes Gi with eij 
whose sum of absolute weight pair is refreshed by the rule: 
Lk+i{e.ij) = L%r^ip{e^j) + /? • LTranp{ey) ■ ff'kiG,)}- Accord- 
ing to the rules of SOSinLeave(D), for any Gi G V{D), 
^k-\-i{Gi) = Psi{Gi{a'j^)). It is convenient to define another 
shorthand operator notation {T'a'){Gi) = P si(G iia')), that 


is T 'al = cTfc+i- By the same way as Lemma 3.1 


3.2 we can prove operator T' is a contraction. 


and Lemma 


4. CASE STUDY 

The details of the example we used can be found in [^. It 
shows a local network connected to Internet (see Figure]^. 
By the assumption that the firewall is unreliable, and the op¬ 
erating system on the machine is insufficiently hardened, the 
attacker has chance to pretend as a root user in web server 
and steal or damage data stored in private file server and 
private workstation. The state set S of example is shown in 



Figure 3: Case study 


Lemma 3.3. Por any Gi G V{D), we have 

I o-fe(Gi) - T'a'k{G,) |< max | Lf (e) - Lf+i(e) | 

eeE(Gi) 

Proof. The proof is similar to that of Lemma [ tT] □ 
Lemma 3.4. T' is a eontraetion. 


TablelH is given in Table and Table [^respectively; 

for convenience, we will mostly refer to the states and ac¬ 
tions using their symbolic number; state transition probabil¬ 
ity is shown in Table [^ in which p{si, 1, 2, si) = P(l|l, 1, 2); 
the immediate payoff to attacker and defender at each state 
is shown in Table in which r“(si,2, •) = P^(l,2, •) and 
r^(si,2, •) = P^(l,2,-), where • means any action available 
at current state. 


Proof. The proof is similar to that of Lemma [3^ □ 


4.1 Modeling for Case study 










We modeling for state si in ComModel as example, then 
we have pDi, pN-^ as follows: 

'Mfc{u).Telld{y).Nil 
pD^ =Tella{x). ^ 'Defd{v).Nil 

vEA^(si) 

pN^ i^^Attc{x).T^.Defd{y).T^.Tri{x,y) 

Tri(x,y) Log{u,v).{if {x — u^y — v) then 

ueAa(si) 
v^A^{s-i ) 

'^[p{si,u,v,Sj)]Rec{r{si, u,v)).(pA^\pDj\pNj) 
jei 

else Nil) 


We find three pairs of states which are probabilistic bisimi¬ 
lar: si 3 ~ si 5 , si 4 ~ si 6 and sir ~ sis- Figure]^ shows the 
ConTS of case study. 


(1,*,0.18,(0,0)) (3,*,0.3,(0,0)) 

(G5r(l.*-0.18,(0,0))-{G2V(l'*’0.3,(10,-10))-^^(l,2,0.1,(10,-10))t7G3 
r 1—(2/3,l,(0,-10) 

3/ 99)) L|.-(1,1,1,(99.-99)) (3,1,1,(0,-10))-1 L(3,i,i,(10,-10' 

\(V2'0-8,(50,-99))_,j^-N„ -(1,2,0.8,(0,0))-(2,'3,i,(0,0))' 

{l,3,0.8,(10,-10))i-( Gd"-—' 

(2/3,3,l,(0,0)MrT_n'3'o5(0-2 



(2,3,0.8,(10,-10)) 

L{|J 


(G 1' (999,-999))^-(Gi J(*,2/3,0.8(0,C 

^ (1.2/3.1.(60.-60) ^ _ 

(1,1,0.5,(30,-10)) 

I-(2/3,1,1,(0,-20)) (1,1,0.5,( 


■(*,1,1, (0,-90))- 

(*,l,l,(30,-60)) (*,2/3,0.8,(0,0)) 
.8(0,0)) 



Figure 4: ConTS of Example 


4.2 Analyzing NES/SOS for Case study 

We implement the algorithms using Java in Eclipse develop¬ 
ment environment on machine with 3.4GHz Inter(R) Core(TM) 
i72.99G RAM. We get two Nash Equlibrium Strategies and 
one Social Optimal strategy for our case study, shown in 
Eigure respectively. 



Figure 5: Nash Equilibrium strategy 1 


4.3 Evaluation 


(1,2,0.5,(0.-10)) 



Figure 6: Nash Equilibrium strategy 2 



Figure 7: Social Optimal strategy 


We compare our results with those obtained in by game- 
theoretic approach: 

(1) We filter the invalid Nash Equilibrium strategy from the 
results in [^. We filter the action pair (4>, Remove^ Sniffer_ 
Deteeor) at state S 3 and the action pair (InstralL Sniffer- 
Deteetor, Remove-CompromisedLaccount- restart-ftpd) at se 
which obtained in the second Nash Equilibrium strategy in 
[12] but have no practical state transition. 

(2) We minimize the state space by probabilistic bisimu¬ 
lation while focuses on the whole state set. Time con¬ 
sumed to compute Nash Equilibrium strategy and Social Op¬ 
timal strategy for this example with our approach is shown 
in Table 2. Although it is incomparable with the time con¬ 
sumed in because of evaluating on different machine 
models, our approach should be faster theoretically. 


ComModel 

Nash Equilibrium 

Social Optimal 

Creation 

strategy 

strategy 

2.8s 

3.7s 

1.4s 


Table 2: Time consumed for example with our ap¬ 
proach 

5 . CONCLUSION 

We proposed a probabilistic value-passing GGS (PVGGS) 
approach for modeling and analyzing a typical network se¬ 
curity scenario with one attacker and one defender which is 
usually modeled by perfect and complete information game. 
Extention of this method might provide uniform framework 
for modelling and analyzing network security scenarios which 
are usually modeled via different games. We designed two 
algorithms for computing Nash Equilibrium strategy and So¬ 
cial Optimal strategy based on this PVGGS approach and 
on graph-theoretic methods. Advantages of these algorithms 












































are also discussed. 
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APPENDIX 

A. PROOFS OF THEOREMS 

Proof of Theorem 13.11 


Proof. As vertex set V is finite, then any infinite execu¬ 
tion 7^^ of Gt is in form of G^e^lGi...{Gk---G(k+m)e(k+m)kGk) 
which means ending with a cycle starting with Gk, and m is 
the number of vertex on this cycle except Gk^ then we have 


PF^{7V^) = Ll,,,p{eu) + /3 • PT“(^,[1]) 

= L’\ve^p{eil) + ... + /3'' • Pf “(7ri[fc]) 

= ^ + y .PF“(7r,[fc]) 

where A = L'iYgip{eii) + ... + /3^ ^ ■ L'iYgip{e(k-i)k) 
PF‘‘{ni[k]) = L'^gip{ek(k+i)) + ■■■ + • L‘\ygip{e(k+m)k) 

+ ^”>+1 . B + ^2m+2 


= {B 


1-y 




-)h 


I _ ^(m + 1) 

where B = L%^^ip{ek(k+i)) + ••• + • L%^eip{e(k+m)k) 

, 1 _ Q{rn+l)h 

lim = lim ^ )) 

h^oo h^oo 1 — 


= apis'" -B - lim 
= A + B 


1 - ' 

j^k 


I — ^(m+l) 


□ 


Proof of Lemma [3T] 

Proof. Assuming without loss of generality, ak{Gi) = 
(Lfc(ei)“,Lfc(ei)'^) and Tak{Gi) = (Lfc+i(e 2 )“, Lfe+i(e 2 )‘*), 
where ei, 62 G E{Gi). Let L^(ei) = a, L^{e 2 ) — ^fc+i(ei) — 
a and L^pi{e 2 ) = 6 ', where a, a', 6 , h' are positive number, 
case 1: 

According to the rules of Backlnd(), we have a < b and 
h' < a'. If the first inequality in lemma doesn’t hold, then 
we have | a — b' | > | a —a' \ and | a — b' | > | b — b' |, then we get 
{b' — a'){b' + a') > 2a{b' — a') and (a — b){a-\-b) > 2b'{a — b) 
which deduce a — b > a' — b', contradiction, 
case 2: L^Act{ei) ^ L^Act{e 2 ) 

Let us define two conditions: 

Cond 1: on kth iteration, 62 is kept by step ( 2 ) of Backlnd(). 
Cond 2 : on (k+l)th iteration, ei is kept by step ( 2 ) of 

Backlnd(). 

There are four subcases to be considered: 
case 2.1: both Cond 1 and Cond 2 

According to the rules of Backlnd(), we have a > b and 
b' > a'. If I a — 5' I > I a — a' I and | a — b' | > | b — b' |, then 
we get b — a > b' — a , contradiction. 
case 2.2: not Cond 2 but Cond 1 

According to the rules of Backlnd(), 3e' with L\ct{^') — 
LActi^i)- Assuming Ll{e') = c and Lt+Ae') = c\ then we 
have c > a > b, a > c and b' > c . If | a — b' |>| a —a \ and 
I a — b' I>I b — b' I, then we have {b' — a'){b'Fa') > 2a{b' — a') 
and a b > 2b'. U b' > a' > c , it is trivial to get contra¬ 
diction; If c < b' < a', then we have 2b' < b' a' < 2a and 
2c > a + 6 > 25' > 2c', then we have b' < a and c > c . If 
I a — b' I > I c — c I, then we have a — c > b' — c ^ contradiction; 


If b' — a!^ contradiction. 

case 2.3: not Cond 1 but Cond 2 

According to the rules of Backlnd(), 3e' with LaciW) — 
L\ct{^ 2 )- proof is similar to case 2.2. 
case 2.4'. neither Cond 1 nor Cond 2 

According to the rules of Backlnd(), 3e', e" with L\ct{^') — 
L\cAe") — L\ct{^ 2 )- Assuming Ll(e') = c 
and Ltp,{e') = c',L^(e") = d and L^+i(e") = d', then we 
have d < a < c, d < b, a' > c and c < b' < d'. If 
I a — b' I > I a — a' \ and | a — b' | > | b — b' |, then we have 
(5'— a')(5'+ a') > 2a{b' — a) and (a —5)(a+ 5) > 2b'(a — b). 
If a > 5 and a' > 5', then we have c < c and a > 5', and if 
I a — b' |>| c—c I, then we a — c > b' — c , contradiction; 
If a < 5 and a' > b' or a > b and a' < b', it is trivial to get 
contradiction; If a < 5 and a' < b', then we get d' > d and 
a < b', and if | a — b' | > | d — d' |, then we get b' — d' > a — d, 
contradiction. 

Proof for second inequality is similar. We skip the details. 

□ 


Proof for Theorem 13.31 


Proof. We need to prove two issues: 

1. NESinLeave(H) is terminated. 

The way to prove termination of NESinLeave(D) is to 
prove 3k that after kth iteration, WGi, Ppk{Gi) = Ppk+i{Gi). 
According to Lemma [3. 2 | trivial; 

2. The result of NESinLeave(T>) is NESof D. MGi G V(T>), 

assuming tt^ whose first edge is e is the execution of Gi 
based on the result obtained by NESinLeave(D), we need 
to prove Trf is NEE of Gi coinductively. As Trf is ended by a 
cycle, we just need to prove any e on Trf, e G E{Gj), is the 
first edge of NEE of Gj. We prove edge e of Gi as exam¬ 
ple. If Trf is not NEE of Gi, according to the definition of 
NEE, there exists Trf satisfying: (1) PT^(Trf ) > PP^(Trf) 
where or ( 2 ) PE^nf) > PE^{nt) where 

e' = arg max PE^iirf ), and both of them are con- 

e"eE^{G„e') ^ ' 

tradicted with the rules in Backlnd(). 

□ 

B. TABLES OF CASE STUDY 

To make paper self-contained, we list the data related in 
example created in p^ . 


State number 

State name 

1 

N ormal-operation 

2 

Httpd^attacked 

3 

Ftp_attacked 

4 

Ftpd-attacked detect or 

5 

HttpdJiacked 

6 

Ftpd-hacked 

7 

W ehsite^def aced 

8 

W ebseversnif fer 

9 

W ebseversnif fer-detector 

10 

W ebsever_DOSA 

11 

W ebsever-DOS-2 

12 

N etw or k shut down 

13 

Filesever-hacked 

14 

Filesever^datastolen 

15 

W orkstationJiacked 

16 

W orkstation-datastolen-1 

17 

Filesever^datastolen 

18 

W orkstation-datastolen-2 


Table 3: Network state 









State no.\ 
Action no. 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 


Install_Snif f er_ 
Detctor 

RemoveSnif fer^ 
Detctor 

Remove-Compromised- 

account-restart-httpd 

Removc-Compromised- 

account-restart-ftpd 

Restore-websitc-remove- 

compromised-account 

0 

Removesniffer-and- 
C ompromised-account 
Removc-virus-and- 
C ompromised-account 
Removc-virus-and- 
Compromised-accoun 
Removc-virus-and- 
C ompromised-account 

4> 

Removc-snif fer-and 
-C ompromised-account 


Install-snif fer- 
detector 

Install-snif fer- 
detector 


18 


Table 4: Defender’s action set 


R\l) = 


R\2) = 


R\3) = 


R\4) = 


R\5) = 


R\6) = 


15 

0 

0 

0 


0 

0 

o' 

J 

■-99 

0 

o' 

16 

Removc-snif fer-and 

0 

(f> 

R\7) = 

0 

0 

0 

fi"(7) = 

-99 

0 

0 


-C ompromised-account 

J. 

J. 

J. 

0 

0 

0 

-99 

0 

0 


10 10 10 

10 10 10 

0 0 0 

o' 


0 0 
0 0 
0 0 
0 0 
0 0 
0 0 
20 
0 0 
0 0 
99 50 

10 0 
0 
0 

10 0 
10 0 


0 
0 

o' 

0 
0 

10 ' 10 


R‘^(1) = -R^il) 


R^{2) = R^{2) 


0 
0 

99' 
10 
10 0 
0 10 ' 

0 
0 


R^{3) = 


R\A) = 


R\5) = 


-10 -10 -20 
-10 -10 0 
-10 -10 0 
-20 -10 -lO' 


-10 

-10 


-99 -99 -99 

10 10 -10 
-10 -10 0 


R‘^(6) = -R^i6) 


State no.\ 
Action no. 

1 

2 

3 

1 

Attack-httpd 

Attack ftpd 

0 

2 

Continuc- 

attacking 

0 

0 

3 

Continuc- 

attacking 

0 

0 

4 

Continuc- 

attacking 

0 

0 

5 

Defacc- 

website 

Install- 

sniffer 

0 

6 

Install- 

sniffer 

0 

0 

7 

0 

0 

0 

8 

Run-DOS 

Crack-fileserver- 

Crack-W or k station 


-Virus 

root-password 

-root-pas sword 

9 

0 

0 

0 

10 

0 

0 

0 

11 

0 

0 

0 

12 

0 

0 

0 

13 

Capture 

-data 

0 

0 

14 

Shutdown 

-network 

0 

0 

15 

Capture 

-data 

0 

0 

16 

Shutdown 

-network 

0 

0 

17 

0 

0 

0 

18 

0 

0 

0 



30 

30 

30" 






rUs) = 

50 

50 

50 

R‘^(8) = 

= - 

R\8) 




50 

50 

50 








-20 

0 


o' 






R\9) = 


-20 

0 


0 

R^(9) = 

--R\9) 





-20 

0 


0 








'30 

0 

o' 






R^(10) = 


30 

0 

0 

R^(IO) 

= - 

-R^(10) 




30 

0 

0 








"30 

0 

o' 



'-60 

0 

o' 

R\ll) = 


30 

0 

0 

i?2(ll) 

= 

-60 

0 

0 



30 

0 

0 



-60 

0 

0 



'o 

0 

o' 




■-90 

0 

o' 

R^(12) = 


0 

0 

0 


R^il2) 

= 

-90 

0 

0 



0 

0 

0 




-90 

0 

0 



■999 

0 


o' 






R^(13) = 


999 

0 


0 

R^(13) 

= - 

-it:^(i3) 




999 

0 


0 








■30 

60 


6( 

)' 


'-10 

-60 

R\14) = 


0 

0 


0 

i?2(14) 

= 

-20 

0 




0 

0 


0 



-20 

0 




■999 

0 


o' 






R^(15) = 


999 

0 


0 

i?2(15) 

= - 

-i?^(13) 




999 

0 


0 








■30 

60 


6( 

)1 


'-10 

-60 

R^(16) = 


0 

0 


0 

R‘^(16) 

= 

-20 

0 




0 

0 


0 



-20 

0 




'o 

0 

o' 







R^(17) = 


0 

0 

0 


i?2(17) 

= i?^(17) 





0 

0 

0 









'o 

0 

o' 







R^(18) = 


0 

0 

0 


R^(18) 

= it:^(i8) 





0 

0 

0 








-60 

0 

0 


-60 

0 

0 


Table 6: 
fender 


Immediate payoff to Attacker and De- 


Table 5: Attacker’s action set 



State 1 State 2 State 3 

p(2|l, 1, •) = 1/3 p(2|2, 1, •) = 0.5/3 p(3|3, 1,2) = 0.5 

p(3|l, 1, 2) = 1/3 p(5|2, 1, •) = 0.5/3 p(3|3, 1, 3) = 0.5 

p(l|l,3,-) = 1/3 p(l|2,2,-) = 1 p(6|3,l,2) = 0.5 

p(l|2,3, •) = 1 p(6|3, 1,3) = 0.5 

p(4|3,l,l) = 1 

State 4 State 5 State 6 

p(l|4, 2,1) = 1 p(7|5, 1,3) = 0.8 p(8|6, 1,3) = 0.8 

p(l|4, 3, 1) = 1 p(8|5, 2, 3) = 0.8 p(9|6, 1, 2) = 0.8 

p(3|4, 1, 1) = 1 p(9|5, 1, 2) = 0.8 pIi\6, 2, 3) = 1 

p(4|4, 1,2) = 1 p(l|5,3, 1) = 1 p(l|6,3, 1) = 1 

p(4|4, 1,3) = 1 p(l|5,l,l) = l p(6|6,2,3) = l 

p(6|6,3,3) = 1 

State 7 State 8 State 9 

p(l\7, •, 1) = 1 p(10|8, 1, •) = 1/3 p(l|9, •, 1) = 1 

p(7|7, •, 2) = 0.9 p(13|8, 2, •) = 0.3 

p(7|7,-,3) = 0.9 p(15|8,3, •) = 0.3 

State 10 State 11 State 12 

p(l|10, •, 1) = 1 p(l|ll, •, 1) = 1 p(l|12, 1, •) = 1 

p(ll|10, •, 2) = 0.8 p(12|ll, •, 2) = 0.8 p(12|12, •, 2) = 0.9 

p(ll|l0,-,3) = 0.8 p(12|ll,-,3) = 0.8 p(12|l2,-,3) = 0.9 

State 13 State 14 State 15 

p(14|13, 1, •) = 1 p(12|14, 1, 2) = 1 p(16|15, 1, •) = 1 

p(12|l4, 1,3) = 1 
p(17|l4, 2, 1) = 1 
p(17|l4, 3, 1) = 1 
p(12|l4, 1, 1) = 0.5 
p(17|l4, 1, 1) = 0.5 

State 16 State 17 State 18 

p(12|16, 1, 2) = 1 p(17|17, •, •) = 0.9 p(18|18, •, •) = 0.9 

p(12|l6, 1,3) = 1 

p(18|l6, 2, 1) = 1 

^(18116, 3, 1) = 1 

p(12|l6, 1, 1) = 0.5 

j3(18|l6, 1, 1) - 0.5_ 


Table 7: State transition probabilities 


C. NOTATION INDEX 

Abs, 5 NESinNonLeave(), 7 

AlgNESO, 5 PF^^iwi), 4 

AlgSOSO, 5 PF^^ini), 4 

Backlnd(), 6 Ppn{Gi), 6 

ComModel, 3 Psn{Gi), 6 

ConTS, 4 PrePro(), 7 

D, 5 PreProSO, 7 

execution, 4 PVCCSr, 2 

Ln{e), 6 RefN(), 6 

LS(e), 6 RefBO, 6 

Leave, 5 SOE, 5 

LocSoOpO, 6 SOS, 5 

NEE, 5 SOSinLeaveO, 6 

NES, 5 SOSinNonLeaveO, 7 

NESinLeaveO, 6 V(D), 5 








